Papers with small source LM
Augmentation-Adapted Retriever Improves Generalization of Language Models as Generic Plug-In (2023.acl-long)
Copied to clipboard
| Challenge: | Prior work on retrieval augmentation fine-tuned the retriever and the LM, making them closely coupled. |
| Approach: | They propose a generic retrieval plug-in that can be used to fine-tune retrieval augmentation and a LM to learn a user's preferences. |
| Outcome: | The proposed retriever improves the generalization of large language models on the MMLU and PopQA datasets by learning LM’s preferences from a known source LM . |